A Majorization-minimization Approach to Variable Selection Using Spike and Slab Priors
نویسندگان
چکیده
We develop a method to carry out MAP estimation for a class of Bayesian regression models in which coefficients are assigned with Gaussian-based spike and slab priors. The objective function in the corresponding optimization problem has a Lagrangian form in that regression coefficients are regularized by a mixture of squared l2 and l0 norms. A tight approximation to the l0 norm using majorizationminimization techniques is derived, and a coordinate descent algorithm in conjunction with a soft-thresholding scheme is used in searching for the optimizer of the approximate objective. Simulation studies show that the proposed method can lead to more accurate variable selection than other benchmark methods. Theoretical results show that under regular conditions, sign consistency can be established, even when the Irrepresentable Condition is violated. Results on posterior model consistency and estimation consistency, and an extension to parameter estimation in the generalized linear models are provided.
منابع مشابه
Spike and Slab Variable Selection: Frequentist and Bayesian Strategies
Variable selection in the linear regression model takes many apparent faces from both frequentist and Bayesian standpoints. In this paper we introduce a variable selection method referred to as a rescaled spike and slab model. We study the importance of prior hierarchical specifications and draw connections to frequentist generalized ridge regression estimation. Specifically, we study the usefu...
متن کاملDeterminantal Priors for Variable Selection A priori basate sul determinante per la scelta delle variabili
Determinantal point processes (DPPs) provide a probabilistic formalism for modeling repulsive distributions over subsets. Such priors encourage diversity between selected items through the introduction of a kernel matrix that determines which items are similar and therefore less likely to appear together. We investigate the usefulness of such priors in the context of spike-and-slab variable sel...
متن کاملSpike-and-Slab Dirichlet Process Mixture Models
In this paper, Spike-and-Slab Dirichlet Process (SS-DP) priors are introduced and discussed for non-parametric Bayesian modeling and inference, especially in the mixture models context. Specifying a spike-and-slab base measure for DP priors combines the merits of Dirichlet process and spike-and-slab priors and serves as a flexible approach in Bayesian model selection and averaging. Computationa...
متن کاملBayesian model selection for logistic regression models with random intercept
Data, collected to model risk of an interesting event, often have a multilevel structure as patients are clustered within larger units, e.g. clinical centers. Risk of the event is usually modeled using a logistic regression model, with a random intercept to control for heterogeneity among clusters. Model specification requires to decide which regressors have a non-negligible effect, and hence, ...
متن کاملSparse Bayesian Variable Selection for the Identification of Antigenic Variability in the Foot-and-Mouth Disease Virus
Vaccines created from closely related viruses are vital for offering protection against newly emerging strains. For Foot-and-Mouth disease virus (FMDV), where multiple serotypes co-circulate, testing large numbers of vaccines can be infeasible. Therefore the development of an in silico predictor of crossprotection between strains is important to help optimise vaccine choice. Here we describe a ...
متن کامل